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Introduction 

Breast  cancer  is  one  of  the  leading  causes  of  death  among  women.  However,  there  is  clear 
evidence  that  early  diagnosis  and  subsequent  treatment  can  significantly  improve  the  chance  of 
survival  for  patients  with  breast  cancer.*-^ 

Mammography  has  become  one  of  the  major  diagnostic  procedures  with  a  proven  capability 
for  detecting  early-stage,  clinically  occult  breast  cancers.^-*  However,  breast  cancers  in  their 
early  stage  are  small  and  frequently  their  radiographic  appearance  differs  only  subtly  from  that 
of  normal  tissue  or  benign  abnormalities.  Because  of  this  subtlety,  the  potential  for 
misclassification  by  radiologists  is  substantial.  Only  10-30%  of  cases  that  have 
mammographically  suspicious  findings  and  are  subjected  to  biopsy  prove  to  be  malignant.^ 

On  the  other  hand,  approximately  10-30%  of  patients  with  breast  cancer  are  misdiagnosed  by 
mammography  (have  the  cancer  missed  or  not  detected  on  their  mammograms). 

Besides  the  subtle  nature  of  radiographic  lesions  associated  with  breast  cancer,  many  errors  in 
radiological  diagnoses  can  be  attributed  to  human  factors  such  as  subjective  or  varying 
decision  criteria,  distraction  by  other  image  features,  and  simple  oversight.^^-i'^  Studies 
suggest  that  these  errors  may  occur  even  with  experienced  radiologists.^®’*^  These  errors  may 
be  reduced  by  the  use  of  automated  detection  schemes  that  can  locate  and  classify  possible 
lesions,  thereby  alerting  the  radiologist  to  examine  these  areas  with  particular  caution. 
Moreover,  the  automated  detection  schemes  can  serve  as  a  "second  radiologist",  similar  to  the 
double  reading  by  two  radiologists  that  is  commonly  practiced  in  diagnostic  radiology  to 
increase  diagnostic  efficacy. 

Microcalcifications  are  commonly  considered  to  be  important  signs  of  breast  cancer.  It  has 
been  reported  that  30-50%  of  breast  cancers  detected  radiographically  demonstrate 
microcalcifications  on  mammograms.^'^'^®  Up  to  90%  of  cases  of  ductal  carcinoma  in  situ 
present  with  microcalcifications.^®  The  correlation  between  the  presence  of 
microcalcifications  and  the  presence  of  breast  cancer  suggests  that  accurate  detection  of 
microcalcifications  will  improve  the  efficacy  of  mammography  as  a  diagnostic  procedure. 

Microcalcifications  occur  in  malignant  and  benign  conditions.  Some  microcalcifications  are 
characteristically  benign  or  are  associated  with  a  benign  process.  For  example,  calcified 
fibroadenomas  have  a  typical  "popcorn"  configuration  appearing  coarse  and  solitary.  Milk  of 
calcium  demonstrates  sedimentation.^’  Vascular  calcifications  have  a  tram  track  appearance, 
typical  of  vascular  calcifications  seen  in  other  areas  of  the  body.  Dermal  calcifications  tend  to 
be  smooth  and  round  with  lucent  centers.  Secretory  calcifications  are  thick,  smooth,  cigar¬ 
shaped,  and  usually  non-branching.  Features  supporting  benignity  include  uniform  size  and 
density  of  the  calcium  flecks,  as  is  seen  in  sclerosing  adenosis.’®  Furthermore,  benign 
microcalcifications  tend  to  be  uniformly  dense  or  scattered,  without  a  segmental  or  linear 
distribution.’^ 

Some  microcalcifications  associated  with  malignancy  have  a  typically  granular  or  linear 
appearance.  They  usually  occur  in  clusters  consisting  of  greater  than  15  particles.®®  The 
particle  size  is  small  (less  than  1  mm)  and  the  shape  is  irregular.®*  Some  clusters  of 
microcalcifications  have  neither  the  typically  benign  nor  typically  malignant  configurations 
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described  above.  These  "indeterminate"  microcalcifications  present  a  significant  diagnostic 
problem  and  require  careful  analysis. 

The  number  of  microcalcifications  per  cm^  has  been  shown  to  be  the  most  important  predictor 
of  malignancy,  with  clusters  consisting  of  less  than  10  microcalcifications  per  cm^  having  a 
high  chance  of  benignity.  Clusters  consisting  of  microcalcifications  numbering  greater  than  15 
per  cm2  h^ve  a  higher  chance  of  mahgnancy.^^ 

The  task  of  detection  and  classification  of  microcalcifications  for  the  diagnosis  of  breast 
cancer  is  a  difficult  one.  The  inability  to  correctly  predict  cancer  is  not  only  due  to  the  overlap 
in  appearance  between  microcalcifications  associated  with  benign  and  malignant  conditions, 
dense  breasts,  improper  technical  factors  or  simple  oversight  by  radiologists  may  contribute  to 
the  failure  to  detect  microcalcifications.  Differing  levels  of  confidence  and  training  among 
interpreting  radiologists  may  lead  to  inconsistent  recommendations  for  management. 

Radiologists  classify  breast  microcalcifications  into  one  of  three  groups;  benign,  hkely 
malignant,  and  indeterminate.  Most  patients  with  indeterminate  types  of  calcifications 
undergo  a  breast  biopsy  to  exclude  cancer.  Any  method  that  would  correcdy  classify  benign 
types  of  calcifications  previously  considered  indeterminate  would  decrease  the  frequency  of 
biopsy  and  therefore  the  cost  of  detection  of  breast  cancer. 

Several  investigators  have  been  developing  computer  programs  for  the  automated  detection  of 
microcalcifications  on  mammograms.^^-^^  Chan  et  al.  showed  that  the  computer  program  can 
detect  subtle  microcalcifications  that  may  be  missed  by  radiologists,  mdicating  that  it  is  a 
promising  approach  to  the  automated  detection  of  microcalcifications.^'^  More  recently,  Wuet 
al.  apphed  an  artificial  neural  network  (ANN)  to  detect  microcalcifications.^*  The  ANN, 
trained  by  using  the  power  spectrum  of  regions  of  interests  (ROI)  containing 
microcalcifications,  was  able  to  eliminate  50%  of  false-positive  detections  of  a  rule-based 
scheme^’ while  preserving  more  than  95%  of  the  true-positive  detections.  The  neural 
network  achieved  an  Az  value  of  0.85  for  the  detection  of  clustered  microcalcifications. 
Several  other  computer  schemes  for  detection  of  microcalcifications  were  also  reported  by 
Astley  et  al.,^  based  on  likelihood  estimators,  by  Grimaud  et  using  mathematical 
morphology  tools,  and  by  Karssemeijer,^^  using  a  stochastic  method  based  on  Bayesian 
decision  theory. 

As  stated  earlier,  microcalcifications  can  be  associated  with  either  benign  or  malignant 
processes.  It  is  important  to  distinguish  different  types  of  microcalcifications  after  they  have 
been  identified  by  a  detection  scheme.  Accurate  classification  of  microcalcifications  into 
benign  and  malignant  groups  would  help  improve  the  sensitivity  of  the  diagnosis  as  well  as 
reduce  the  number  of  unnecessary  biopsies. 

As  the  first  step  in  the  process  of  developing  an  automated  computer  scheme  for  classification 
of  microcalcifications,  a  neural  network  system  was  developed  to  classify  microcalcifications 
in  the  radiographs  of  biopsy  specimens.  Classification  of  microcalcifications  in  radiographs  of 
biopsy  specimens  is  an  “idealized”  situation. 
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In  biopsy  specimens,  underlying  tissue  around  microcalcifications  is  less  than  that  present  in 
normal  mammograms.  Therefore,  the  scatter  radiation  recorded  on  films  is  reduced,  resulting 
in  improved  contrast.  Higher  dose  and  geometrical  magnification  can  also  be  used  to  obtain 
radiographs  of  biopsy  specimens  as  compared  with  regular  mammograms.  Less  magnification 
results  in  less  geometrical  unsharpness.  Higher  exposure  can  be  used  to  achieve  greater  signal 
to  noise  ratio  and  thereby  improve  image  quality  of  radiographs. 

Therefore,  microcalcifications  in  the  radiographs  of  biopsy  specimens  are  more  clearly 
represented  than  those  in  regular  mammograms.  After  we  can  successfully  apply  our 
algorithm  to  classify  microcalcifications  in  radiographs  of  specimens,  we  will  make  necessary 
adjustments  to  apply  the  algorithm  to  the  regular  mammograms. 

In  recent  years,  rapid  progress  of  research  on  artificial  neural  networks  (ANN)'^^  h^s  been 
reported  extensively  in  the  field  of  computer  science  and  many  applied  fields.  Neural 
networks  address  detection,  classification,  and  decision-making  problems  not  by  pre-specified 
"conventional"  algorithms,  but  rather  by  "learning"  from  examples  presented  repeatedly.  The 
popularity  of  neural  networks  is  primarily  due  to  their  apparent  ability  to  make  decisions  and 
draw  conclusions  when  presented  with  complex,  noisy,  or  peutial  information  and  to  adapt 
their  behavior  to  the  properties  of  the  training  data.  Neural  networks  are  capable  of  parallel¬ 
processing  a  large  amount  of  information  simultaneously  and  have  been  shown^'*’^^  to  be  a 
useful  tool  for  pattern  recognition  in  fields  where  conventional  algorithmic  approaches  and 
rule-based  expert  systems  may  not  be  successful. 
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Body  of  Report 


METHODS 


Algorithms  for  classification  of  microcalcifications  have  been  developed  in  two  stages.  First,  a 
convolution  neural  network  based  method  was  developed  to  classify  microcalcifications  using 
radiographs  of  pathological  specimen.  Second,  an  image  feature  based  method  was  developed 
to  classify  microcalcifications  using  real  mammograms.  The  image  features  used  and  optimal 
structure  of  the  backpropagation  neural  network  were  determined  by  using  genetic  algorithm. 

Classification  of  microcalcifications  in  pathological  specimen 

The  overall  approach  for  the  classification  of  microcalcifications  using  a  CNN  system  is 
shown  in  Figure  1.  The  radiographs  of  pathological  specimen  are  digitized  by  a  high 
resolution  digitizer.  Regions  of  interest  (ROI)  containing  microcalcifications  are  manually 
selected.  These  ROIs  are  preprocessed  and  used  as  input  to  the  CNN  system.  Finally,  the 
classification  results  are  examined  by  the  ROC  analysis. 


Figure  1.  Overall  approach  for  the  classification  of  microcalcifications  using  CNN. 
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The  classification  of  microcalcifications  is  based  primarily  on  the  fact  that  microcalcifications 
associated  with  malignant  processes  generally  have  more  irregular  shapes  with  fuzzy  and 
spiculated  boundaries  and  are  less  uniform  in  density  and  size.  They  are  usually  grouped  into 
multi-particle  clusters.  The  microcalcifications  associated  with  benign  processes,  on  the  other 
hand,  usually  tend  to  have  smoother  and  well-defined  boundaries,  rounded  shape,  and  uniform 
densities  and  sizes.  The  neural  network  system  will  be  trained  to  recognize  the  characteristics 
of  each  type  of  microcalcifications. 

The  CNN  is  based  on  the  network  structure  of  Fukushima’s  Neocognitron^®  which  is  designed 
to  simulate  the  vision  of  vertebrate  animals.  The  structure  of  CNN  used  in  this  study 
resembles  a  simplified  Neocognitron.  A  two-dimensional  convolution  operation  from  the 
input  layer  to  the  hidden  layer  is  employed  to  simulate  radiologists’  viewing  of  a  suspected 
area,  liie  CNN  has  the  ability  to  process  and  recognize  two  dimensional  image  patterns  and 
has  been  shown  to  be  an  effective  tool  in  image  processing  and  pattern  recognition.'*’^  '*^ 

Acquisition  of  Mammograms 

The  selected  radiographs  of  breast  biopsy  specimen  are  digitized  with  an  image  resolution  of 
21pm  X  21pm  per  pixel  by  a  CCD  camera  digitizer  (DBA  Systems  Inc.).  With  high  resolution 
digitization,  the  morphological  information  of  microcalcifications  can  be  preserved,  which 
enables  the  neural  network  system  to  differentiate  different  types  of  microcalcifications  on  the 
basis  of  their  geometrical  shapes  and  density  patterns.  Figure  2(a)  shows  a  cluster  of 
microcalcifications  in  original  radiographs  of  pathological  specimen.  Shown  in  Figure  2(b)  are 
clustered  microcalcifications  after  being  digitized  with  the  high  resolution  digitizer.  The 
shapes  and  density  patterns  of  the  microcalcifications  are  better  defined  in  Fig.  2(b)  than  those 
shown  in  Fig.  2(a). 


(a)  (b) 

Figure  2.  Microcalcifications  shown  in  an  original  radiograph  of  pathological  specimen 
(a)  and  shown  in  a  radiograph  digitized  with  a  high  resolution  (21  jim  x  21  p,m) 
digitizer  (b). 
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Convolution  Neural  Network 

The  structure  of  the  CNN^’  is  shown  in  Figure  3.  The  input  to  the  CNN  are  ROIs  of  matrix 
size  of  64  x  64  pixels,  containing  benign  or  malignant  type  of  microcalcifications.  Only  one 
hidden  layer  is  used  in  this  study.  The  connections  between  input  and  hidden  layer  are 
grouped  into  seven  different  kernels  based  on  the  structure  of  Fukushima's  Neocognitron.^^’^® 
There  are  two  output  units  in  the  output  layer,  with  each  unit  corresponding  to  a  benign  or 
malignant  class  of  microcalcifications.  The  hidden  layer  and  the  output  layers  are  fully 
connected. 


Input  Hidden  Output 


Figure  3.  Structure  of  a  convolution  neural  network  used  for  the  classification  of 

microcalcifications. 

The  training  algorithm  of  the  CNN  is  similar  to  that  of  a  backpropagation  neural  network,  in 
which  backpropagation  and  the  generalized  delta  rule  are  used  in  the  training  process.^i  The 
input  signals  are  now  two  dimensional  images.  The  weights  are  all  arranged  in  the 
convolution  kernels.  In  the  feed-forward  propagation,  the  output  of  Ith  layer  are  first 
convoluted  with  weight  filters.  The  sum  of  the  convolution  is  then  added  by  a  bias  term  to 
form  the  net  input  to  the  next  layer. 

N‘;\x,y) = [1] 

p=l 

where  N‘p\x,y)  is  the  net  input  to  the  unit  (x,y)  in  layer  1+1 ,  Op(x,y)  is  the  output  of  the 
unit  in  layer  I,  ^  (x,y)  is  a  weight  kernel,  and  is  a  bias  term  in  layer  1+1 .  In  the 

notation,  layer  number  /  =  (1,  2, ...,  L),  cluster  number  in  the  Ith  layer  p  =  (1,  2, ...,  P^),  and 
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cluster  number  in  the  (/+i)th  layer  q  =  (1,  2, Note  that  *  denotes  discrete 
convolution. 


We  can  then  rewrite  equation  [1]  as 

0‘;\x,y)  =  f(N‘;\x,y)), 
where /is  the  activation  function 


1  +  exp(-x)  ’ 

In  the  error  backpropagation,  the  weights  are  modified,  similar  to  that  in  Eqn.[2],  as  the 
following. 


[3] 


[4] 


to  minimize  the  error  function, 

^  ^  E  ( Tix, y)  -  of  (x,  y)f ,  [6] 

where  T(x,y)  is  the  target  output. 

In  the  training  process  of  the  CNN,  each  image  block  is  rotated  and  reflected  such  that  the 
number  of  training  data  are  increased  eight  fold.  The  rotation  and  reflection  represent 
different  orientations  of  microcalcifications  in  mammograms.  The  trakung  with  additional 
orientation  can  effectively  make  the  CNN  rotational  invariant. 
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Classification  of  microcalcifications  in  real  manunograms 

An  alternative  approach  was  employed  to  classify  microcalcifications  of  in  clinical 
mammograms.  Microcalcifications  can  be  characterized  by  a  number  of  quantitative  variables 
describing  the  underlying  key  features  of  a  suspicious  region  such  as  the  size,  shape,  and 
number  of  microcalcifications  in  a  cluster.  These  features  are  calculated  by  an  automated 
extraction  scheme  for  each  of  the  selected  regions.  The  features  are  then  used  as  input  to  a 
backpropagation  neural  network  to  make  a  decision  regarding  the  probabihty  of  maUgnancy  of 
a  selected  region.  The  initial  selection  of  image  features  set  was  a  rough  estimation  that  may 
include  redundant  and  non-discriminant  features.  A  genetic  algorithm  was  employed  to  select 
an  optimal  image  feature  set  from  the  initial  feature  set  and  select  an  optimized  structure  of 
the  neural  network  for  the  optimal  input  features.  Finally,  the  performance  of  optimized 
neural  network  is  evaluated  using  the  selected  sub-features.  The  performance  of  neural 
network  is  then  compared  with  that  of  radiologists  in  classifying  the  clusters  of 
microcalcifications. 

A  selected  region  is  a  256  x  256  matrk  from  a  mammogram  that  is  digitized  at  50  microns  per 
pixel  with  a  Lumisys  scanner.  Figure  4  shows  two  regions  of  interest  (ROI)  selected  from 
original  mammograms:  (a)  benign  and  (b)  malignant.  The  background  trend  in  a  selected 
region  is  eliminated  by  an  adaptive  trend  correction  technique.  The  trend-corrected  image  is 
then  binarized  by  taking  the  top  level  of  32  quantization  levels.  A  series  of  pre-processing 
techniques  that  include  open  and  dilation  operations,  logic  AND,  and  labeling  operation  are 
applied  to  the  binary  image  to  eliminate  artifacts  while  preserving  the  original  shapes  of 
microcalcifications.  Image  features  are  extracted  based  on  the  pre-processed  image  and  two 
morphologically  filtered  images.  Nineteen  image  features  are  extracted  for  each  cluster  of 
microcalcifications  characterizing  the  size,  number  of  microcalcification  in  the  cluster,  and 
shape. 


(a)  (b) 


Figure  4  Two  ROIs  of  size  256  x  256  selected  from  original  mammgrams:  (a)  benign 
and  (b)  malignant 
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Pre-Processing 

The  pre-processing  algorithm  is  demonstrated  in  Figure  5.  The  purpose  of  pre-processing  is 
to  obtain  a  binary  image  of  microcalcifications.  The  pixels  representing  microcalcifications 
have  values  of  one  and  the  rest  have  values  of  zero.  We  need  only  binary  image  because 
image  features  we  use  are  not  based  on  density  but  rather  based  on  only  size,  shape, 
clustering,  and  number  of  microcalcifications  in  a  cluster. 


Figure  5  Pre-Processing  of  Selected  Region  of  a  Mammogram 
a.  Adaptive  background  trend  correction 

A  average  filter  of  kernel  size  of  23  x  23  pixels  is  used  to  remove  the  background  trend  in  a 
selected  region.  At  each  pixel  in  the  selected  region,  an  average  value  is  calculated  for  the 
surrounding  23  x  23  pixels  and  subtracted  from  the  original  pixel  value.  This  process  is 
repeated  for  all  the  pixels  in  the  selected  region.  The  adaptive  technique  is  used  to  isolate 
microcalcifications  from  their  immediate  surroundings  rather  than  from  a  different  region  that 
may  have  a  different  density  level.  As  long  as  the  sizes  of  microcalcifications  remain  small  and 
microcalcifications  are  not  clustered  too  close  to  each  other,  this  method  will  be  able  to 
preserve  the  original  shapes  of  microcalcifications.  The  kernel  size  of  23  pixels  is  equivalent 
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of  approximately  1.2mm.  Therefore,  microcalcifications  that  are  larger  than  1.2mm  in  size  are 
eliminated  by  the  background  correction  process. 

b.  Adaptive  thresholding 

After  background  correction,  a  threshold  is  applied  to  the  image  such  that  only  2048  pixels  of 
highest  intensity  (pixel  value)  are  kept  (assigned  pixel  value  1)  and  the  rest  pixels  are  assigned 
pixel  values  of  0.  The  number  of  pixels  that  are  kept  after  thresholding  is  determined 
empirically  that  the  sizes  of  microcalcifications  in  the  threshold  image  are  similar  to  those  in 
the  original  image. 

c.  Opening  operation 

A  morphological  Opening  operation  with  3x3  kernel  size  is  applied  to  the  binary  image  to 
remove  line  artifact.  The  Opening  is  a  combination  of  an  erosion  followed  by  a  dilation. 

d.  Dilation 

A  morphological  Dilation  operation  is  applied  after  the  OPEN  operation  to  fill  in  the  small 
holes  near  the  boundary  of  an  object. 

e.  AND  operation 

To  preserve  the  shape  of  microcalcifications,  an  AND  operation  between  the  original  binary 
image  and  the  image  after  dilation  is  performed. 

f.  Labeling 

A  labeling  process  is  applied  after  the  AND  operation  to  eliminate  small  objects  that  still 
remain  in  the  image  that  are  less  than  5  pixels  in  size.  Any  object  that  is  that  small  is  unlikely 
to  be  an  microcalcification. 

Image  Feature  Extraction 

Three  images  are  used  in  feature  extraction.  The  three  images  are:  Image  1,  the  preprocessed 
image;  Image2,  dilation  of  Image  1  by  a  5  x  5  kernel;  Image3,  dilation  of  Image  1  by  a  25  x  25 
kernel.  The  process  is  demonstrated  in  Figure  6. 

1 .  Number  of  microcalcifications  N  from  Image  1 

2.  Area  (number  of  pixels)  in  Image  1  S 1 

3.  Area  (number  of  pixels)  in  Image2  S2 

4.  Area  (number  of  pixels)  in  Image3  S3 

5 .  (S2-S 1  )/N,  a  measure  of  average  number  of  irregular  pixels 
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6.  (S3-S1)/N,  a  measure  of  average  distance  between  microcalcifications  in  a  cluster 

7.  (S2-S 1 ),  total  number  of  irregular  pixels 

8.  (S3-S1),  a  measure  of  average  distance  between  microcalcifications  in  a  cluster 

9.  VI ,  size  of  the  largest  microcalcification  in  Image  1  -  size  of  average  microcalcification  in 
Image  1 

10.  V2,  size  of  the  average  microcalcification  in  Imagel  -  size  of  the  smallest 
microcalcification  in  Imagel 

1 1.  V3,  size  of  the  largest  microcalcification  in  Imagel 

12.  V4,  size  of  average  microcalcification  in  Imagel 

13.  V5,  size  of  smallest  microcalcification  in  Imagel 

14.  El,  ellipticity  of  the  largest  microcalcification  on  Imagel 

15.  WAE,  weighted  average  ellipticity  of  microcalcifications  on  Imagel 

16.  Ax,  length  of  semi-major  axes  of  the  fitted  ellipse  of  a  cluster  on  Imagel 

17.  Bx,  length  of  semi-minor  axes  of  the  fitted  ellipse  of  a  cluster  on  Imagel 

18. Ax/Bx 

19.  Ax  X  Bx 

A  total  of  19  image  features  are  extracted  for  candidate  cluster  and  used  as  input  to  a 
backpropagation  neural  network. 


(a)  (b)  (c)  (d)  (e) 


Figure  6  (a)  Original  gray  scale  image,  (b)  Binary  image,  (c)  Best-fit  ellipse  and  its  semi¬ 

major  axis  A^  and  semi-minor  axis  B^,  (d)  Dilation  with  5x5  kernel,  and  (e) 
Dilation  with  25  x  25  kernel. 
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Genetic  Algorithms 

Genetic  algorithms  are  directly  modeled  after  biological  systems  and  behavior.  All  the  values 
of  a  structure  represent  the  neural  network  characteristics  that  uniquely  define  a  candidate 
solution  in  the  space  of  possible  solutions.  Utilizing  genetic  algorithms,  the  Genetic 
Supervisor  evolves  successor  populations,  or  generations,  from  a  limited  population  of  initial 
candidate  solutions.  It  does  this  by  treating  the  inclusion  or  exclusion  of  each  column  of  data 
in  the  fuU  Input  colunm  set  as  features;  the  number  of  layers  and  the  number  of  neurons  per 
layers  as  features;  and  the  control  parameters  of  each  neural  network  as  features. 

These  features  are  then  varied  in  each  new  generation  with  the  resulting  structure  evaluated  in 
terms  of  neural  network  fitness.  Each  stmcture  in  the  generation  is  evaluated  and  judged  by 
either  the  lowest  RMS  Error  achieved  after  a  fixed  number  of  epochs  or  by  the  number  of 
epochs  taken  to  achieve  a  minimum  point  in  RMNS  Error.  These  two  measures  represent 
neural  networks  that  train  to  minimal  error  or  neural  networks  which  train  with  minimal 
epochs.  These  criteria  can  be  applied  to  the  set  of  training  cases  or  the  set  of  test  cases. 

If  the  structure  representing  a  neural  network  successfully  meets  the  fimess  criteria  selected, 
then  the  values  of  its  features  will  be  retained  and  bred  with  other  structures.  For  each 
generation,  the  Genetic  Supervisor  generates  a  population  of  structures  in  on  of  two  ways. 

All  the  structures  of  an  initial  generation  and  a  certain  number  of  structures  in  subsequent 
generations  are  created  with  features  set  to  random  values  constrained  within  specified  limits. 
Subsequent  generations  are  created  by  cross-breeding  the  strings  of  successful  structures  or 
occasional  mutations  of  randomly  selected  features  of  successful  structures.  Some  or  many  of 
the  weakest  structure  may  be  culled,  these  are  replaced  with  new  structures. 

Through  this  evolution-like  process,  an  optimal  neural  network  can  be  developed.  Note, 
however,  that  this  process  requires  the  training  of  many  versions  of  the  neural  network  to 
determine  an  optimal  one;  for  neural  network  models  that  have  large  network  configurations 
or  have  large  data  sets  this  can  be  a  lengthy  process  -  but  so  can  biological  evolution! 

A  system  consisting  of  a  Backpropagation  neural  network  and  a  Genetic  Supervisor  is 
developed  to  classify  the  extracted  image  features.  The  Genetic  Supervisor  attempts  to  select 
the  best  subset  of  image  features  from  the  provided  input  set,  configure  the  best  neural 
network  structure,  and  adjust  the  parameters  of  the  network  for  optimum  performance.  The 
performance  of  the  neural  network  system  is  evaluated  by  ROC  analysis. 
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Reduction  of  Patient  Call  Backs 

A  third  set  of  images  will  be  selected  at  GUMC  which  are  images  of  patients  that  are  called 
back  after  screening  exam  to  follow  up.  A  majority  of  the  called  back  patients  are  benign. 

The  objective  is  to  reduce  the  number  of  call  back  patients  are  definitely  benign  and  therefore 
reduce  the  overall  cost  of  breast  cancer  diagnosis  practice. 

Database 

Pathological  Specimen  Images 

Eighty  regions  of  interest  (ROI)  that  contain  clustered  microcalcifications  (40  benign  and  40 
malignant)  are  selected  from  60  digitized  radiographs  of  pathological  specimen  in  this  study. 
Figure  7  shows  all  of  the  80  ROIs  selected  in  the  database.  There  are  substantial  variations  in 
size  among  benign  or  malignant  microcalcifications.  The  information  concerning  the 
classification  of  microcalcifications  ("truth")  are  obtained  from  the  results  of  biopsy 
examination.  Background  trend  correction  is  employed  to  remove  the  non-uniform 
background  structure  in  different  ROIs. 

Clinical  Mammograms 

Two  sets  of  clinical  mammograms  were  collected.  The  first  set  contains  70  cases  of  clusters 
of  microcalcifications  and  the  second  set  contains  62  cases  of  microcalcifications.  The  two 
sets  cases  were  selected  from  two  different  sources  of  medical  institutions.  The 
microcalcifications  in  both  sets  were  rated  subtle  by  radiologists.  The  diagnostic  truth  of  all  of 
the  cases  in  the  selected  sets  had  been  verified  by  biopsy. 
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(b) 

Figure  7  Database  for  the  training  and  testing  of  the  CNN;  (a)  40  ROIs  containing  benign 
clustered  microcalcifications  and  (b)  40  ROIs  containing  malignant  clustered 
microcalcifications . 
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RESULTS 

images  of  pathological  Specimen 
Jackknife  Method 

A  jackknife  method  was  employed  to  evaluate  the  performance  of  the  CNN.  In  the  jackknife 
method,  half  of  the  ROIs  were  randomly  selected  from  the  database  of  80  ROIs.  These  ROIs 
were  used  to  train  the  convolution  neural  network.  The  other  half  of  the  ROIs  were  then  used 
to  test  the  performance  of  the  CNN.  By  choosing  different  random  samples  from  the 
database,  the  jackknife  test  can  be  repeated  to  generate  multiple  test  output  and  provide  a 
better  estimate  of  the  true  performance  of  the  CNN  in  classifying  benign  and  malignant 
clusters  of  microcalcifications. 

ROC  Analysis 

The  output  values  from  the  two  output  units  were  examined  by  using  Receiver  Operating 
Characteristic  (ROC)  analysis.^2,53  xhe  LABROC4  algorithm^^  developed  by  Metz  et  al.  was 
used  to  fit  ROC  curves  to  the  continuous  data  from  the  output  of  CNN.  The  area  under  the 
ROC  curve  (Az)  was  used  as  an  overall  measure  of  diagnostic  performance.  The  result  from 

each  jackknife  test  was  analyzed  individually  by  using  ROC  analysis.  Ten  jackknife  tests  were 
performed.  A  final  ROC  curve  was  obtained  by  averaging  the  results  from  the  10  jackknife 
tests,  as  shown  in  Figure  8.  The  CNN  system  performed  very  well  in  classifying  benign  and 
malignant  clusters  of  microcalcifications,  achieving  an  Az  value  of  0.90. 

Potential  Application  in  Recommending  Courses  of  Action 

A  potential  application  of  CNN  is  to  classify  microcalcifications  into  groups  of  definitely 
benign  and  possibly  malignant.  By  applying  a  low  threshold  level  to  the  output  values  of  the 
CNN,  we  can  make  CNN  a  classifier  that  is  not  very  specific  but  with  100%  sensitivity. 

With  such  a  classifier,  some  benign  microcalcifications  may  be  classified  as  possibly  malignant, 
but  all  of  the  microcalcifications  classified  as  benign  are  definitely  negative.  Thus,  radiologists 
can  ignore  the  clusters  of  microcalcifications  that  are  classified  as  benign  and  only  concentrate 
on  those  that  are  classified  as  possibly  malignant.  As  a  result,  the  time  radiologists  spend 
reading  mammograms  can  be  reduced  and  detection  efficacy  of  breast  cancer  can  be  expected 
to  improve. 
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Figure  8  ROC  analysis  of  the  performance  of  CNN  in  classifying  benign  and  malignant 
microcalcifications . 

The  ability  of  CNN  to  serve  as  a  classifier  to  eliminate  benign  microcalcifications  can  be 
demonstrated  in  Table  I.  In  each  of  the  jackknife  tests,  we  set  threshold  levels  of  the  output 
of  CNN  such  that  all  of  the  malignant  ROIs  are  to  be  called  positive  by  the  computer  system 
and  calculate  the  number  of  the  benign  ROIs  that  can  be  called  negative  (i.e.,  have  output 
values  below  the  threshold  level  for  positive  ROIs). 

Table  I  shows  the  results  for  each  individual  Jackknife  test  as  well  as  the  averaged  results  of 
the  10  jackknife  tests  (second  column).  The  CNN  can  identify,  on  average,  approximately 
42%  of  the  benign  clusters  of  microcalcifications  with  100%  sensitivity  (without  missing  any 
malignant  clusters).  Therefore,  if  this  CNN  system  were  used  to  help  radiologists  in  detecting 
malignant  microcalcifications,  radiologists  would  only  need  to  examine  about  half  of  the 
detected  microcalcifications. 

As  discussed  earlier,  studies  have  shown  that  approximately  10-30%  of  breast  cancers  are 
missed  by  mammography  and  only  10-30%  of  biopsy  cases  recommended  by  mammography 
are  actually  malignant.  As  a  comparison  to  the  reported  performance  of  radiologists  in  breast 
cancer  diagnosis,  we  also  listed  in  Table  I  (third  and  fourth  columns)  the  average  specificities 
and  positive  predictive  values,  defined  as  the  portion  of  malignant  cases  among  the  biopsied 
cases. 
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With  a  sensitivity  of  80%,  approximately  the  same  level  of  sensitivity  reported  by  average 
radiologists,  the  neural  network  system  achieved  a  positive  predictive  value  for  malignancy  of 
84%,  compared  with  10-30%  achieved  by  radiologists.  Therefore,  the  CNN  appears  to  be  a 
very  promising  tool  for  assisting  radiologists  in  making  decisions  for  the  diagnosis  of  breast 
cancer. 

Real  mammograms 

Genetic  algorithms  are  employed  to  optimize  the  structure  of  a  neural  network  and  select  the 
best  performing  subset  from  the  initial  image  features.  Additional  cases  of  patients  that  have 
been  recalled  for  further  examination  are  being  collected  at  GUMC.  The  performance  of  the 
neural  network  system  based  on  these  cases  will  help  determining  the  usefulness  of  the  system 
in  reducing  unnecessary  biopsies. 

Optimized  Feature  Set  bv  Genetic  Algorithms 

Genetic  Algorithms  were  apphed  to  the  first  subset  of  seventy  cases  selected  from  the 
database.  Two  different  sets  of  image  features,  each  consisting  of  six  features,  were  selected 
as  a  result  of  the  optimization  process.  The  neural  network  performed  equally  well  with  the 
two  selected  subsets. 

Feature  Set  #1: 

1.  V3,  size  of  the  largest  microcalcification  in  Image  1 

2.  V4,  size  of  average  microcalcification  in  Imagel 

3.  V5,  size  of  smallest  microcalcification  in  Imagel 

4.  El ,  elhpticity  of  the  largest  microcalcification  on  Imagel 

5.  WAE,  weighted  average  ellipticity  of  microcalcifications  on  Imagel 

6.  Ax,  length  of  semi-major  axes  of  the  fitted  ellipse  of  a  cluster  on  Imagel 

Feature  Set  #2: 

1 .  El,  elhpticity  of  the  largest  microcalcification  on  Image  1 

2.  WAE,  weighted  average  eUipticity  of  microcalcifications  on  Imagel 

3.  Ax,  length  of  semi-major  axes  of  the  fitted  eUipse  of  a  cluster  on  Imagel 

4.  Bx,  length  of  semi-minor  axes  of  the  fitted  eUipse  of  a  cluster  on  Imagel 

5.  V3-V4 

6.  V4-V5 

The  optimized  parameters  for  the  neural  network  are:  learning  rate  0.7  ~  0.95  and  momentum 
0.02  ~  0.06. 
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We  evaluated  the  performance  of  the  optimized  neural  network  by  using  round-robin  method 
and  ROC  analysis.  Figure  9  shows  the  ROC  curve  for  the  classification  of  microcalcifications 
based  on  extracted  image  features. 


False-Positive  Fraction 

Figure  9  Performance  of  feature-based  neural  network  in  classification  of  microcalcifications 

The  system  of  neural  network  and  genetic  algorithms  improved  performance  over  our 
previous  TRBF  neural  network^^.  The  neural  network  system  was  able  to  classify  benign  and 
malignant  microcalcifications  at  a  level  favorably  compared  to  experienced  radiologists.  On 
the  same  database  evaluated,  the  ROC  curve  for  an  experienced  radiologist  yielded  an  Az  of 
0.54,  while  the  Az  of  the  ROC  curve  for  the  neural  network  with  optimized  structure  is  0.68. 
The  use  of  the  neural  network  system  can  be  used  to  help  radiologists  reducing  the  number 
biopsies  in  clinical  applications. 

DISCUSSIONS 

It  is  important  to  note  that  the  CNN  is  designed  as  an  automated  classifier  of 
microcalcifications  for  the  diagnosis  of  breast  cancer.  It  will  be  used  in  conjunction  with  other 
schemes  for  the  detection  of  microcalcifications  in  digital  mammograms.  Once 
microcalcifications  are  detected,  the  CNN  will  be  applied  to  classify  them  into  benign 
(negative)  and  malignant  (positive)  groups.  Radiologists  can  ignore  the  microcalcifications 
that  are  classified  into  the  benign  group  and  examine  those  that  are  classified  as  malignant  to 
decide  whether  to  recommend  biopsy  or  short  term  follow-up  exams. 
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Table  I.  Application  of  CNN  in  Reconunending  Courses  of  Action 


Jackknife  Test 

Correctly  idertitled 
negative  cases  at 
1(X1%  sensitivity 

False  Positives  at 
90%  sensitivity 

False  Positives  at 
80%  sensitivity 

16 

2 

2 

12 

7 

2 

15 

2 

0 

2 

6 

6 

11 

3 

1 

4 

2 

2 

7 

4 

2 

7 

8 

8 

3 

5 

5 

6 

5 

2 

8.3 

4.4 

3 

42% 

78% 

85% 

I-.;'-''-' 

63% 

80% 

84% 

*  Positive  Predictive  Value — Defined  as  the  portion  of  the  actually  positive  cases  among 
the  cases  diagnosed  that  are  classified  as  positive  by  a  diagnostic  system. 

The  results  discussed  are  based  on  radiographs  of  biopsy  specimen  of  microcalcifications. 

The  specimen  images  have,  in  general,  better  image  quality  and  greater  signal-to-noise  ratio 
than  the  regular  mammograms.  The  radiographs  digitized  with  high  resolution  digitizers 
provide  the  morphological  information  of  individual  microcalcifications  that  makes  the 
classification  of  microcalcifications  into  benign  and  malignant  groups  possible.  The  CNN 
system  will  need  to  be  tested  on  regular  mammograms.  Some  parameters  of  CNN  may  need 
to  be  fine  tuned  when  applied  to  regular  mammograms  and  the  CNN  system  may  not  achieve 
the  same  performance  level  as  it  did  in  this  study. 

Both  a  large  training  and  testing  database  are  necessary  in  order  to  train  and  evaluate  the 
performance  of  the  neural  network  sufficiently  and  reliably.  We  will  be  expanding  our 
database  significantly  in  the  future.  To  further  improve  the  accuracy  of  the  classification,  a 
hybrid  neural  network  (HNN)^^  will  also  be  employed  to  classify  microcalcifications  based  on 
the  input  of  both  image  data  and  image  features^"^  that  will  be  automatically  extracted. 

The  evaluation  of  feature  based  neural  network  with  Generic  Algorithm  was  based  on  a  very 
difficult  data  set  which  was  different  from  what  was  used  for  evaluation  of  image  based  neural 
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networks.  The  microcalcification  clusters  in  the  data  set  for  feature  based  neural  network  are 
very  subtle  and  an  experienced  radiologist  only  scored  an  Az  of  0.54  in  classifying  between 
benign  and  malignant  clusters.  The  size  of  the  database  is  also  small.  We  are  currently  in  the 
process  of  expanding  our  database  to  include  more  mammograms  from  different  hospitals. 
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Conclusions 

We  have  demonstrated  that  the  convolution  neural  networks  can  be  an  effective  tool  in  the 
diagnosis  of  breast  cancer.  The  results  obtained  in  this  study  are  very  promising,  even  though 
they  were  based  on  a  relatively  small  training  and  testing  database.  These  results  indicate  the 
potential  usefulness  of  CNN  in  classification  of  microcalcifications  in  digital  mammograms. 
An  extensive  clinical  test  of  our  developed  system  using  real  mammograms  will  be  needed  to 
determine  the  clinical  applicabiUty. 

Genetic  algorithms  are  an  effective  tool  to  select  optimal  input  features  and  structure  of  a 
backpropagation  neural  network.  The  neural  network,  combined  with  genetic  algorithms,  is 
able  to  effectively  classify  benign  and  malignant  microcalcifications.  The  results  of  the  neural 
network  system  can  be  used  to  help  reducing  the  number  of  benign  biopsies. 
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